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1.  Introduction. 


In  this  report  we  propose  a  method  for  recursive  evaluation  of 
certain  probabilities  associated  with  two  classes  of  finite  homogeneous 
Markov  chains.  These  chains  are  next  of  kin  to  Bernoulli  random  walks 
with  reflecting  and  absorbing  barriers.  They  may  be  roughly  character¬ 
ized  by  the  following  four  properties: 

(1)  Except  for  the  barrier  stites,  (one-step)  transitions  from  each 
state  can  be  made  to  exactly  two  other  states. 

(2)  The  states  are  divided  into  two  subsets,  S  and  S^,  plus  an 
initial  state  in  the  absorbing  case.  State!  in  S  communicate 

with  states  in  S,  only  via  a  pair  of  states,  one  in  each  subset. 

D 

(3)  Except  for  the  absorbing  states,  the  probability  of  transition 

among  states  in  and  among  states  in  has  only  two  values, 

Pa»  1  -  pa  and  p^,  1  -  p^  respectively. 

(4)  If  states  in  each  of  the  two  subsets  are  ordered  such  that  the 
transitions  with  probability  p  are  all  between  adjacent  states  in 
one  direction  then  the  transitions  with  probability  1  -  p  are  all 
in  the  opposite  direction,  but  not  necessarily  to  the  adjacent  state. 

A  glance  at  Figures  1  and  3  may  help  to  reveal  the  structure  of  a  typical 

member  of  those  two  classes. 

We  are  interested  in  the  ratio  of  limiting  probabilities  of  a 

chain  being  in  the  subset  S  and  the  subset  S,  .  In  sections  2  and  3 

a  b 

we  present  a  recursive  formulae  for  evaluating  these  probabilities.  Subse¬ 
quent  examples  show  that  the  computation  is  considerably  simpler  than  the 
classical  method  of  solving  systems  of  linear  equations.  Our  method  involves 
nothing  but  repeated  substitution  and  is  easy  to  perform  and  program  even 
for  a  large  number  of  states. 

The  need  for  studying  these  ratios  arises  in  problems  connected 
with  finite  automata  with  binary  inputs  and  outputs  driven  by  a  Bernoulli 
sequence.  These,  in  turn,  appear  in  the  so-called  finite  memory  problems 
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(References  [1]  through  [A]),  which  are  currently  receiving  considerable 
attention  in  literature. 

The  reason  for  writing  this  report  is  twofold.  First  since  the 
proofs  of  our  formulae  (Sections  4  and  5)  are  basically  algebraic  and  thus 
rather  long  it  is  usually  necessary  to  condense  the  proof  when  the  formula 
is  used  as  a  lemma.  Hence  we  wanted  to  have  the  proof  documented  in  full 
detail  for  reference.  Next,  it  is  conceivable  that  Markov  chains  of  the 
type  studied  here  may  be  encountered  in  various  stochastic  models.  Hence, 
the  second  purpose  of  this  report  is  to  provide  an  access  to  our  results 
to  other  workers  in  the  general  area  of  stochastic  modelling. 

To  this  we  would  like  to  add  that  the  two  formulae  can  probably  be 
generalized  in  several  directions.  For  instance,  inspection  of  the  proofs 
Indicate  that  the  same  method  could  still  be  used  to  establish  similar 
formulae  for  a  larger  class  of  chains,  namely  without  the  property  (3) 
above. 

The  part  on  ergodlc  chains  (Sections  2  and  4)  and  the  part  on 
absorbing  chains  (Sections  3  and  5)  can  be  read  independently. 
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Let  r  -  {r  (2)  ,r  (3) and  r.  -  {r.  (2)  ,r.  (3)  , . . . }  be  two 

a  3  3  *T)  D  D 

sequences  of  positive  integers  such  that  1  <  r  (i)  <  i,  1  £  r.  (i)  <  i, 

a  b 

i  ■  2,3,...  With  each  such  pair  (r  ,r.  )  we  associate  a  class 


E(r  ,r.)  -  {M _ :  n-1,2,..,  ;  m-1,2,...} 

— a  — D  n,m 

of  finite  ergodic  Markrv  chains.  The  chain  M  has  n  +  m  states 

n,m 

which  are  divided  into  two  subsets  S  and  S.  with  n  and  m  states 

a  b 

respectively . 

We  label  the  states  in  by  (i,a),  i  -  l,...,n,  and  the 

states  in  S,  by  (i,b),  i  -  l,...,m.  The  transition  probabilities  are 

D 

as  follows: 

P((i,a)  -*•  (i+1 ,a) )  -  p  ,  i  -  1 . n-1 , 

a 

P((n,a)  -*■  (m,b) )  -  p  , 

O 

P((i,b)  -  (i+l,b))  -  pb,  i  -  1 . m-1, 

P((m,b)  -*■  (n,a) )  -  pb> 

P((i,a)  -*■  (r  (i)  ,a) )  -  q  ,  i  -  2,...,n, 

a  a 

P((l,a)  -*•  (1  ,a)  )  -  q  , 

a 

P((i,b)  (rb(i),b))  -  qb,  i  -  2,...,m, 

P((l,b)  •>  (l,b))  -  qfe. 

Here  0<p  <1,  0<p.  <1,  q  -1-p,  q.  -  1  -  p.  .  All  other 

ra  rb  na  ra  b  b 

transition  probabilities  are  zero.  The  transition  diagram  is  depicted  in 


Figure  1. 
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Figure  I 
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Proposition  1:  Let  M  €  E(r  ,r.  ).  let  u(s).  s  €  S  US.  be  its 
- -  n  ,m  —a  b  a  b 

stationary  distribution,  let 


u(S  )  =  V  u(s)  and  u(S  )  =  £  u(s) 


s€S 


s€Sl 


be  the  stationary  probabilities  of  the  chain  being  in  S  and  S. 

a  b 

respectively.  Then 


U(S  )  p"  A 

a  b  n 

U(S  )  n  B  ’ 

b  p  m 


(2.1; 


where  A^  and  B^  are  polynomials  in  p^  and  p^  respectively  satisfying 
the  recurrence  relations 


Vi  * ’ 


n 

a  ‘a 


r  ,  n-i.  . 

L  Pa  *  A  *  1 , 

l-r  (n+1)  a  1 

a 


n  -  1,2 .  (2.2) 


Bm*l  ‘  Pb  +  ’b  J  B1  ’  K 

Ist  r,  (nH-1 ) 

b 


m  =  1,2, 


(2.3) 


Hence,  both  A  and  B  have  integral  coefficients  and  are  of  degree 
n  m 

less  than  n  and  m,  respectively. 

(For  the  proof  see  Section  4.) 


Example  1:  Let  (r  ,r^) 

be  given  by  the 

following  table 

i 

r  (D  . 

rh(1) 

a 

D 

2 

1 

l 

3 

1 

l 

4 

2 

3 

5 

• 

3 

6 

• 

• 

6 


Figure  2 


let  n  *  4,  m  ■  5.  The  transition  diagram  of  this  9 -state  chain  is 
in  Figure  3. 
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First  evaluate  A.  and  Bc.  From  (2.2)  we  have 
A  5 


A  »  p3  +  q  (A  p3_2+A  p3-3)  , 
A  a  a  L  a  J  a 


A,  ■  p2  +  q  A  p2'1  +  q  A.p2"2  , 
3  a  a  la  ^a  2a 


A„  -  p  +  q  A  p  1-1 
L  a  a  l  a 


Aj  "  1  » 


and  substituting  from  the  bottom  to  the  top  gives 


Aj  ■  1  . 


*2  ‘  pa  *  «.  '  1  • 


*3  ‘  K  +  V,  +  ' 


A.  -  p3  +  q  p  +  q  p2  +  q2p  +  q2  , 
A  ra  a  a  ^a  a  ^ara  na 


or  by  substituting  q  ■  1  -  p 


\  “  P. 


P2  +  1. 


bc  ■  p!*  +  qw(BiPw~3+B/Pu"^)  . 

j  D  D  J  D  H  D 


BA  “  Pb  +  qbB3Pb"3  * 


b,  -  p2  +  q.  (B,Pv’l4-B,Pw”2) 

J  D  DID  ZD 


Similarly  from  (2.3) 
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b2  “  pb  *  VlPb_1  ’ 


B1  "  l* 


and  again  substituting 


B2  *  Pb  +  qb  "  1  ’ 

B3  "  Pb  +  Vb  +  qb  * 

B4  "  Pb  +  qbPb  +  qbPb  +  qb  ’ 

B5  “  Pb  +  qbPb  +  qbPb  +  qbPb 

+  qbPb  +  qbPb  +  qbPb  +  qb  ’ 
or  substituting  for  q,  -  1  -  p, 

D  D 

B5  -  P3  -  pb  +  1. 


u(s  )  p*  pi  -  p2  +  1 
"  pj  Pb  ■  pb  + 1  * 


Hence,  from  (2.1) 
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3.  Absorbing  Chains. 

Let  r  -  {r  (l),r  (2),...}  and  r.  -  {r.  (1) ,r  (2),...}  be  two 
a  a  a  “t>  b  b 

sequences  of  nonnegative  Integers  such  that 

0  *  ra(i)  <  1,  0  *  rb(l)  <1,  1  -  1,2,... 

With  each  such  a  pair  (r  ,r.  )  we  associate  a  class 

- a  - to 

A(r  ,r.)  -  {M  :  n-1,2,...  ;  m«l,2,...} 

—a  - t>  n,m 

of  finite  absorbing  Markov  chains.  The  chain  M  has  n  +  m  +  1  states, 

n,m 

two  of  them  absorbing  and  the  rest  transient.  One  state  is  always 

designated  as  an  initial  state  while  the  remaining  n  +  m  states  are 

divided  into  two  subsets  and  with  n  and  m  states  respectively, 

each  containing  one  of  the  two  absorbing  states. 

We  label  the  states  in  by  (i,a)  i  -  l,...,n  with  (n,a) 

absorbing,  and  the  states  in  S,  by  (i,b),  i  ■  l,...,m  with  (m,b) 

b 

absorbing.  The  initial  state  is  labeled  (0,a)  or  (0,b)  or  just  0 
as  needed . 

The  transition  probabilities  are  as  follows: 

P((i,a)  -*■  (i+1  ,a) )  -  p  ,  i  -  l,...,n-l, 

P((n,a)  -*■  (n,a) )  -  1, 

P((i,b)  -*•  (i+l,b))  -  pb,  i  -  1 . m-1, 

P ( (m,b)  -  (m,b) )  -  1, 

P((i,a)  -►  (ra(i),a))  -  qa>  i  -  l,...,n-l, 

i  ®  1 , . . . ,m-l , 


P((i,b)  -  (rb(i),b))  -  qb. 
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p(0  -*■  (l,a))  = 


P„  +  Pv 


p (o  -  (i,b)>  =  — 1 . 

Pa  +  Pb 

Here  0  <  p  <  1,  0  <  p  <  1,  q  =  1  -  p  ,  q,  *  1  -  p.  .  All  other 

a  Dear  b 

transition  probabilities  are  zei-e.  The  transition  diagram  is  depicted 
in  Figure  2. 


Proposition:  Let  M  6  A(r  ,r,  )  ,  let  ir(a)  and  ir(b)  be  the  probabilities 
-  n  ,m  —a  - b 

of  absorption  in  the  state  (n,a)  and  (m,b)  respectively,  if  the  initial 
state  is  the  state  0.  Then 


ff(a)  _  ^a 
ir(b)  ~  m 

where  A  and  B  are  polynomials  in 
n  m 

the  recurrence  relations 

n  k-r  (k) 

^n+1  ^  ^a  ,  ^r  (k)Pa 

k=2  a 


m  k-r^(k) 

Bmfl  '  1  -  %  l  Br„(k)pb 
k=2  b 


B 

m 

A  » 
n 

p  and  p 
a  b 


9 


-  0, 


(3.1) 

respectively  satisfying 


n  ■  1,2, ,  (3.2) 


m  ■  1,2, . . .  .  (3.3) 


Hence  both  A  and  B  have  integral  coefficients  and  are  of  degree 
n  m 

less  than  n  and  m  respectively. 

(For  the  proof  see  Section  5.) 
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Example  2:  Let 


(r  ,r,  )  be  given  by  the  following  table 

&  D 


0 

1 

1 

3 

3 


let  n  ■  4,  m  =  5. 
in  Figure  4. 

First  evaluate 


The  transition  diagram  of  this  10-state  chain  is 


and  B^.  From  (3.2)  we  have 


\  - 1  -  vvri+vri>  • 


ai  • 1  -  wr0  • 


A0  -  ° . 


and  substituting  from  the  bottom  to  the  top 


or  substituting  for  q 

d 


Aq  =  0  , 


=  1  , 


1  -  q  p  -  q  p2  , 
na  a  nara 


1  -  Pa 

\  ‘  Pl  '  Pa  +  1- 


Similarly  from  (3.3) 

“s'1-  VBlpb'4Blpb'l+B3pb'1)  • 
b3  =  1  "  ViPb’1  * 


1, 
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and  again  substituting 


or 


B5  "  1  •  h%  ~  *bqb  "  Pbqb  +  pjqj  , 
B5  =  2pb  "  Pb  ‘  Pb  +  l- 


IS*)  m  2pb  ~  pb  ~  ph  +  1 

T(b)  p£  pJ  -  p  +  i 

b  *a  *a 


Hence,  from  (3.1) 
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4 .  Proof  of  Propose  *•  Lon  1. 

Let  P  be  the  transition  probability  matrix  for  the  chain  M  , 

n  ,m 

where  the  first  n  rows  and  columns  correspond  to  states  (1  ,a) , . . . ,  (n,a) 
and  the  following  m  rows  and  columns  to  states  (m,b)  , . . . , (1 ,b) . 

Let  ^  =  (y (1 ,a) , . . . ,y(n,a) ,y(m,b)  , . . . ,y(l ,b))  be  the  stationary 
distribution,  so  that 


E(I-P)  -  0  , 


(4.1) 


where  I  is  the  identity  matrix.  Now  partition  the  matrix  P  into  four 
submatrices 


P  - 


where  P  is  an  n  *  n  matrix 
a 


Row: 
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and  is  an  m  *  m  matrix 

Row: 


Column:  n  i  r.  (i)  1 

D 


n 


i 


r.d) 


2 

1 


Notice  that  each  row  of  these  matrixes  contains  exactly  one  entry  q, 

th 

namely  the  (i,r(i)) —  one,  and  that  the  labelling  of  rows  and  columns 
of  Pb  begins  at  the  lower  right  corner  while  the  labelling  of  Pg  is 
the  usual  one  (beginning  at  the  upper  left  corner) . 

The  off-diagonal  matrices  V#  and  consist  of  all  zeros  except 

for  the  lower-left  corner  entry  of  V  ,  which  is  p  ,  and  the  upper- 

A  A 

right  corner  entry  of  V.  ,  wt  ich  is  p,  . 

b  b 

With  this  partitioning  the  equation  (4.1)  decomposes  into  two 
equations 


l^(I-Pa)  ■  (O,...,0,p(m,b)pb), 

(4.2) 

■  (u(n,a)pa,0,...,0) , 

(4.3) 

where 
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*  (y(l . ,y (n,a) )  , 

and 

^  -  (y(m,b) , . . . ,p(l,b)) . 


Consider  the  matrix  equation  (4.2)  first.  Solving  for  gives 


“  (0,...,0,w(m,b)pb)(I-Pa) 


-1 


or  denoting  a  the  (i,j)—  entry  of  the  inverse  (1-P  )  ^ 
ij  a 


u(i,a)  -  y(m,b)p  a  ,  j  *  1,...  n. 
d  n ,  l 


(4.4) 


By  the  well-known  formula  for  matrix  inversion 


a 


U 


» 


where  |  I  -  P  |  is  the  determinant  of  I  -  P  and  I  1  -  p  |  \ 

A  A  \  K.  |  Jt ) 

is  the  (k,l)—  cofactor  of  I  I  -  P  I.  Hence 

a  1 


u(S  ) 
a 


l 


i-l 


p(i,a) 


p(m,b)pb 

|  I  -  P  f 
'  a  1 


l 


i-l 


I  “  P  ,,  V. 

a  (i,n) 


(4.5) 


Next  let  A  be  the  determinant  of  the  n  *  n  matrix  obtained  from 
n 

I  -  P  by  replacing  the  n^-  column  by  a  column  of  l's, 
a 
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Expanding  A  along  this  last  column  we  obtain 
n 


^  “  I  I  1  -  P  1/4 

n  a  (i,n) 


since  the  (i,n)—  cofactors  of  A  and  I  I  -  P  I  are  identical. 

n  1  a  1 

Thus,  (4.5)  can  be  written  as 


U(S  ) 
a 


u(m,b)pt 
I  -  P 


A  . 
n 


Next  since  the  only  flow  of  probability  between  seta  and  is 

through  states  (n,a)  and  (m,b)  we  must  have 


p(m,b)pb  -  y(n,a)p4 


in  the  stationary  regime.  Using  (4.4)  we  obtain 

I  I  -  P 


u(n,a)  -  p(m,b)p 


a  (n,n) 


I  -  P 


(4.7) 


and  substituting  from  (4.7) 


But 


I  -  P 


I  -  P 


(n,n) 
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I 


I  -  P 


a  ' (n,n) 


which  is  same  as  the  determinant  of  the  (n-1)  *  (n-1)  matrix  1  -  P 

a 

obtained  for  the  chain  ^  €  EO^.r^).  Employing  temporarily  the  super¬ 

script  (n)  for  the  number  of  states  in  S  we  have  a  recurrence  relation 

a 


x(n)  _  p(n) 

a 


p  I 

a 1  a 


and  since 


I<1)  _  p(l) 

a 


p  we  obtain 
*a 


Thus 


j(n)  _  p (n) 

a 


W<Sa)  ■  W(m,b)pb. 

P- 


Now  going  back  to  (4.3)  and  repeating  all  the  steps  above  we  obtain  a 
similar  expression 


20 


w(S.  )  -  y(n,a)p  , 
d  ra  a 


where  B  is  the  determinant  of  order  m 

TL 


m 


•\ 


•  \ 


\ 


pb 


Hence,  using  again  (4.7)  we  have 


y(S  )  p™  A 

a  _  __b  n 

U(S,  )  n  B  ’ 

b  p  ra 
a 

and  it  remains  to  prove  that  Ar  and  B^  satisfy  the  recurrence  relations 
(2.2),  (2.3). 

We  begin  with  the  determinant  A  .  To  evaluate  this  determinant 

n 

let 


IL  **  (i  -  2, . . .  ,n  :  r#(i)  ■  1) 


Notice  that  1^  is  the  set  of  exactly  those  row  indices  1  for  which  the 
(i,l)—  entries  in  (4.6)  are  -q  .  Now  multiply  the  first  row  in  (4.6) 
by  <la/Pa  and  add  it  to  all  rows  such  that  i  €  Ij*  The  determinant 


becomes 
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A 

n 


V\ 


-q  1  -p 


0 


where  we  temporarily  dropped  the  subscript  a  to  ease  the  notation.  The 
entries  in  the  last  column  are  given  by 


1  +  q/p  if  i  6  Ij, 
1  if  i  i  Ij. 


Expanding  this  determinant  along  the 


first  column  we  have 


D‘n>  -  PD<"> . 


where  ■  A  is  the  original  determinant  (4.6)  and 

In 

.001 


D<n)  - 


P  “P 


22 


22 


is  of  order  n  -  1.  Notice  that  the  entries  in  tie  first  colunn  in 
are  -q  only  for  row  indices  i  «  3,...,n  such  that  either  r(l)  •  1 
or  r(i)  -  2.  Hence,  calling  1^  •  U  ■  3,...,n  :  r(i)  <  3),  multiplying 
the  first  row  in  by  p/q  and  adding  to  rovs  with  i  6  I2  this 

determinant  becomes 


P  "P 


t 


(n) 

32 


The  entries  in  the  last  column  are 


(n) 

’3i 


■f 


(n) 

t(n) 

if 

i  e  i 

21 

q 

22 

(n) 

21 

if 

i  £  1 

Expanding  again  along  the  first  column  we  have 


where 
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Proceeding  in  this  fashion  we  obtain  a  sequence  of  determinants 

D(n)  D(n)  D(n) 

»u2  • • * • i 

where  Is  of  order  n  -  k  +  1.  The  entries  t/*V  in  the  last 

k  ki 

column  of  satisfy  the  recurrence  relation 


(4.8) 


where 


t  (n) 
kfl.i 


(n)  ,  £  (n) 

ki  q  kk 

if 

icl 

(n) 

ki 

if 

i  i  I 

(n) 


k  ■  1  ■  k,...,n,  t'  ■  1, 

+  *  ^ 


1^  ■  (1  ■  k+1 . n  :  r(l)  £  k} . 


(4.9) 


Further 


so  that 


nU>  -  nD(n) 
\+l  pDk  * 


D,(n)  -  pn’1D(n)  -  pn*1t(n). 
J  r  n  r  nn 

(n+1) 


(4.10) 


Consider  now  the  determinant  A 
by  using  the  same  sequence  £, 


of  order  n+1  obtained 


(n+1)  _ 
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Applying  the  above  procedure  to  D 


(n+1)  4(n+l)  .  „  , 

'  -A  we  obtain  a  sequence 


(nH)  D(nfl) 

U1  . Vl  * 


(4. 


where  the  determinants  again  satisfy  (4.9)  with  n  replaced  by 

n  +  1.  Arrange  now  the  last  columns  of  the  sequences  (4.8)  and  (4.11) 
into  triangular  arrays  as  follows: 


T(ir*-1) 


(n+1) 

C11 

(n+1)  (n+1) 
c12  ,c22 


(n+1)  (n+1) 
In  ’  2n 

(n+1)  (n+1) 
l,n+r2,n+l 


t(n+l) 

nn 

(n+1)  (n+1) 
n,n+l’  n+1, n+1 


ID 
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Since  for  i  <  n  by  the  definition  of  sets 

i  €  1^°^  if  only  if  i  6 

the  first  n  rows  of  and  are  identical,  i.e. 

i  ■  k, . . .  ,n  ;  k  -  1 . n. 

if  k  *  1, . . .  ,r(n+l) , 

l  t.(nJ‘1)  if  k  -  r (n+l)+l . n+l 

r(n+l) 

In  particular  for  k  -  n  +  1  since  r(rrt-l)  <  n  +  1 


(n)  _  (n+l) 

k±  cki  * 


Next  by  (A. 9) 


(n+l)  _ 

k,n+l 


1+4 

P 


1- 


<nfl>  «  i  + 1  y  t 

n+1*nfl  p  w(ifi) 11 


But  by  (A. 12)  for  1  <  n  +  1 


(n+l)  _  ,(n)  _  (!) 

«  l!  ***  !!  * 


so  that  (A. 13)  becomes 


(n+l) 
n+l, n+l 


n 


l 

!-p(n+l) 


Hence,  by  (A. 10) 


n 

1  +  4  l 

p  l"p(n+l) 


(A. 12) 


(A. 13) 
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or  calling  again 


A  we  have 
n 


n+1 


n  , 
Pa  + 


qa  E  Alp 
a  l-r(n+l)  1 


1.2, , 


where  clearly  ■  1. 

The  recurrence  relation  for  B  is  established  in  exactly  the 

m 

same  fashion. 

Noticing  the  obvious  fact  that  the  polynomials  Aq  and  B^  must 
have  integral  coefficients  completes  the  proof  of  Proposition  1. 
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5.  Proof  of  Proposition  2. 

Notice  first  that  with  0  being  the  initial  state  any  subsequent 
visit  to  this  state  is  a  recurrent  event.  Call  this  event  E^.  Next  call 
the  event  which  occurs  if  the  chain  after  leaving  the  state  0  reaches 
the  absorbing  state  (n,a)  without  any  further  visit  to  state  0.  Similarly, 
define  for  the  absorbing  state  (m,b) .  Now  clearly 


P(EQ)  >  0, 


and  since  the  absorbtion  in  (n,a)  occurs  if  and  only  if  we  have  either 


E  or  RE  or  R.R_E  etc. 
a  0  a  0  0a 


P(Ea) 

1  -  P(E0)  ’ 


and  similarly 


1  -  P(EQ)  ’ 


so  that. 


if(a)  P(Ea} 
w(b)  "  P(Eb)  ' 


(5.1) 


p(E  )  “  ~ ~  P(E^), 

a  Pa  +  Pb  a 


(5.2) 


where  El  is  the  event  which  occurs  if  and  only  if  the  chain  after  leaving 
a 

the  state  (l,a)  reaches  the  absorbing  state  (n,a)  without  ever  visiting 
the  state  0. 


Consider  now  a  subchain  obtained  from  the  chain  M  by 

n  n,m 

making  the  state  0  an  absorbing  state  and  deleting  states  (l,b)  though 
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(m,b) .  The  transition  probability  matrix  for  this  subchain  is  the 
(n+1)  x  (n+1)  matrix 


If  this  subchain  is  started  at  the  state  (l,a)  then  PCE1)  is 

a 

equal  to  the  probability  of  absorption  in  (n,a)  for  this  subchain.  Using 
the  well-known  result  from  the  algebraic  theory  of  Markov  chain  (cf .  [5), 
Theorem  3.3.7)  we  have 


p<Ei) 

a 


Paal,n-1’ 


(5.3) 


where  [a,,]  =  (1-Q  )~1  and  Q  is  the  (n-1)  *  (n-1)  matrix  of  transi- 
ij  a  a 

tion  probabilities  between  transient  states  of  M®,  i.e. 


I 


F 
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By  the  formula  for  matrix  Inversion 


1  -  Q 


l.n-1  |  I  -  Q. 


a1 (n-1,1) 


(5.4) 


where  |  I  —  Q  |  is  the  determinant  of  I  -  Q  and 

d  3 

s  t 

Is  Its  (n-1,1) —  cofactor.  Now 


I  -  Q 


a1 (n-1,1) 


n-2 

p.  * 


and  calling  » 


I  - 


Q  we  obtain  from  (5.2),  (5.3)  and  (5.4) 
a 


P(E.>  ’  T-^Th  *"  • 

ra  rb  n 


(5.5) 
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Following  the  same  procedure  for  the  event  ^  we  obtain 


m 


'<V 


P.  +  Pk  B  ' 
a  bm 


(5.6) 


where  Bm  “  I  1  ~  |  and  Is  the  (m-1)  x  (m-1)  matrix  of  transi¬ 
tion  probabilities  between  transient  states  (l,b) . (m-l,b).  Hence, 

(5.1),  (5.5)  and  (5.6)  yields  (3.1)  and  it  remains  to  establish  the  recur¬ 
rence  relations  for  A  and  B  . 

n  m 

To  do  this  we  evaluate  the  determinant 


where  we  dropped  the  subscript  a  to  ease  the  notation.  Notice  that 
is  of  order  n  -  1  and  that  each  row  but  the  first  has  either  exactly  one 
subdiagonal  entry  equal  to  -q,  namely  the  (i,r(i))“,  or  all  subdia¬ 
gonal  entries  are  zero.  The  former  case  occurs  if  r(l)  >  0  while  the 
latter  if  r(i)  -  0. 

Next  consider  the  determinant  A  of  order  n  obtain  for  the 

n+i 

chain  M  with  the  same  (r  ,£.  ) .  Then 

nrl  ,m  ~ a  — to 
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is  of  order  n  -  1.  Notice  that  the  entry  -q  in  the  last  row  moved  one 
step  to  the  right.  Expanding  D  again  along  the  last  column  gives 


where  D  , 
n-1 


is  of  order  n  -  2 
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Now  repeating  this  the  entry  -q  eventually  (after  n  -  ra(n)  steps) 
reaches  the  diagonal  and  we  have 
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To  Include  the  case  r  (n)  *  0  define  Kn  ■  0.  Then  (5.9)  reduces  to 

a  U 

(5.7).  Finally,  repeatedly  substituting  for  A  in  (5.9)  and  using  the 

n 

obvious  fact  Aj  »  1  ve  obtain  the  recurrence  relation  (3.2).  Notice 

that  (3.2)  holds  also  for  n  ■  1  since  rg(l)  “  0  always. 

The  relation  (3.3)  for  B  is  established  in  exactly  the  same 

m 

feshion  from  |  I  -  | . 

Noticing  the  obvious  fact  that  the  polynomials  A  and  B  must 

n  m 

have  Integral  coefficients  completes  the  proof  of  Proposition  2. 
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